@fgma as #5430 is merged, it is possible to have space (and other special characters) in minio password entered in the install- or upgrade-wizard.
The format of the custom environment-variables has changed and I have updated the changelog in spk/minio/Makefile. Please keep this text when resolving the conflict for this tile.

Sorry, I've made a mistake and messed up the merge.

Sorry, I've made a mistake and messed up the merge.

Yes, I think you still want to update to 2022-10-24T18-35-07Z for compatibility with current package.

Should be OK now.

Anything left here?

Anything left here?

We should add some information to the changelog, regarding that future versions will not be compatible with this one.
And we should know how the update/migration will work.

I've updated the changelog. My idea was to get this update out so everyone has at least this version and new installations will not need a migration. Migration is a separate step for me.

Any ideas on how we should handle this? I don't see any chance doing this automatically because one would need the same amount of temporary storage used by data stored in minio (at least in the biggest bucket) which is not possible to guarantee.

I've upgraded my whole setup to this version for production use. How are we going to continue from here? I'd like to continue upgrading to the latest minio release as soon as possible.

@fgma and @hgy59 is there a hard requirement to #5649 or can this be merged and published as-is?

@th0ma7 Not for me. I've used the manual shared folder creation on my system as a workaround.

@fgma and @hgy59 is there a hard requirement to #5649 or can this be merged and published as-is?

The target of #5649 will be an independent solution to fix the shared folders.
By independent I mean that I will have to create new Makefile variables to not touch existing implementations.
So this is not a blocker for any package that use the old version.
(but AFAICR @fgma wants to have the new version to benefit from the feature to create shares on diferent volumes in advance).

@hgy59 I'm don't really need #5649 but I want to upgrade to the latest minio release which uses a new data format and the release in this PR ist the last one which supports the old format and the new format at the same time. For details have a look at #5549

Anything left for me to do do get this finally merged?

The currently available version has at least two problems:

1. No security updates for over a year.
2. Everyone who stores data in the old data format has to do a manual migration at some point in time. The longer this update takes the more users are affected.

We need a strategy to get this finally resolved.

I may be able to provide a review over the weekend although from a strategy standpoint it needs a volunteer to take it over (unless that would be you, haven't followed this thread in particular)

I can't decide how the migration path should look like. I've opened a thread for discussing the issue almost a year ago which received no reaction at all: #5549

After doing my own migration I'm pretty sure any automation would result in a disaster. It took me multiple weeks for a few terrabytes because tools would timeout all the time and miss file etc. Maybe we need to create a new package or something like that.

Getting this update out would at least would save new users from a migration their data.

So considering the other changes are in (thnx @hgy59) i guess this is a rather trivial update. What I would suggest is adding a wizard with a single page to explain that this will be the LAST version compatible with the older format. You would need to add that as well to the changelog so it is more obvious as info would be hands-on even prior to installing.

On top of that, I'd suggest creating a "new" package for the "new" format, perhaps called minio-mc or something similar to make a distinction between the two. So for the older package you clearly state that this is the last version... then in the new package you explain that this is the newer one.

Guessing there will be issues to be sorted out relatively to ports in usage or else? You could then create a wiki page in spksrc github section to explain how to migrate from minio to minio-mc ...

Otherwise, as-is it looks fine. The above proposal can be done at a later while creating the "new" package where the older one can be re-published with the added wizard and info in the changelog.

My two cents...

I've added a step in the wizard explaining the situation to the user. This ready for me.

After the merge I would create a new package based on a copy of the released version and upgrade it to the current minio release.

@fgma sorry for my late review

Looking at the details of the migration I got some new ideas.

This minio package provides the wizard options to chose the shared folder independent of new installation or package update. In either case, you can use a folder with or without a minio configuration.

Since the current version and the version in this PR supports the legacy FS and the SNSD (Single-Node Single-Drive) mode, we should now use the SNSD-Mode when the shared folder does not contain a minio configuration.

The installer could check whether the shared folder already contains a minio configuration and either configure minio server for legacy FS or SNSD mode.
If the existing configuration is not already SNSD, this Minio version (2022-10-24T18-35-07Z) will display a deprecation warning in the installation wizard, and future versions will refuse the installation without former migration.

I suppose the minio server mode can be recognized with the file .minio.sys/format.json in the shared folder. For the legacy FS mode this looks like:

{
  "version": "1",
  "format": "fs",
  "id": "05583150-3d04-4058-bc30-7e8600932b87",
  "fs": {
    "version": "2"
  }
}

I highly recommend that this update uses the SNSD mode by default so it is useful for the migration of existing installations.

@fgma what do you think about it?

EDIT:
There is no need to configure the minio server for SNSD. When a new shared folder is selected, the the file .minio.sys/format.json looks like this:

{
  "version": "1",
  "format": "xl-single",
  "id": "83ac901d-9fc8-42a5-afd8-dc2e694795bb",
  "xl": {
    "version": "3",
    "this": "46b08f42-a295-4b04-9d43-a8d99dddfc8d",
    "sets": [
      [
        "46b08f42-a295-4b04-9d43-a8d99dddfc8d"
      ]
    ],
    "distributionAlgo": "SIPMOD+PARITY"
  }
}

So for future updates there is no need to create a new package (with a different name), but the installer must refuse the installation/update when the selected shared folder exists and contains an incompatible (legacy FS) configuration.

@fgma The minio console displays the version DEVELOPMENT.GOGET under metrics - info - server.
Isn't there a way to build an official version that displays 2022-10-24T18-35-07Z?

@fgma can you rebase this PR to current master so we have #5649 to apply the redesigned shared folder handling?

@hgy59 I'll try to get the fixes done during the next days.

Concerning bucket versioning: I've verified what you already noticed. When creating a new install even older versions of minio already create SNSD based data by default for a new installation. I've just tested this on version 2022-10-24T18-35-07Z and 2022-09-07T22-25-02Z (current version packaged) using official minio binaries on windows. The issue is mainly a problem for all long term users already having data stored inside minio using quite old releases for the initial install.

How does one gracefully stop an installation? We would need to dynamically detect the data version after the user has selected the storage location and then decide if we can continue or stop with an error. Is there is something similar in the codebase already?

@fgma The minio console displays the version DEVELOPMENT.GOGET under metrics - info - server. Isn't there a way to build an official version that displays 2022-10-24T18-35-07Z?

@fgma can you rebase this PR to current master so we have #5649 to apply the redesigned shared folder handling?

fixed in 6727bb7

Any feedback on this?

How can we move forward with this?

This PR is lacking migration to redesigned shared folder handling.

SERVICE_WIZARD_SHARE = wizard_data_directory
USE_DATA_SHARE_WORKER = yes

should be replaced by

SERVICE_WIZARD_SHARENAME = wizard_data_directory

Then the install_uifile has to be adjusted (remove volume configuration)
and service-setup.sh must use SHARE_PATH variable instead of wizard_data_directory

probably it is better to rename the variable wizard_data_directory to wizard_share_name to reflect the fact that it is the name of the share and not a full path anymore.

finally upgrade_uifile.sh and service-setup.sh have to be adjusted for the shared folder.

There is a good example of modifications in spk/owncloud/ by @mreid-tt

How does one gracefully stop an installation? We would need to dynamically detect the data version after the user has selected the storage location and then decide if we can continue or stop with an error. Is there is something similar in the codebase already?

Sorry for my late reply.
There are the functions validate_preinst and validate_preupgrade that can be used for this kind of validation.

Just echo an "ERROR TEXT" and use "exit 1" in a validate function to refuse the installation or upgrade of the package (you find commented examples in spk/demoservice/src/service-setup.sh).

All such functions (service_* and validate_*) are called after wizard pages are completed and the installation or upgrade is started.

@fgma glad to tell you that this old pending PR is about to be ready to merge.

The redesinged shared folder handling is implemented and wizards are adjusted.

I shortened the Information about migration in the wizards and created a wiki page showing the detailed migration procedure.
(https://github.com/SynoCommunity/spksrc/wiki/MinIO)

Future versions only need a validation of the data format to refuse installation/upgrade if FS format is found in the shared folder.

I know you can't test this since you've already migrated your installation, but having a second set of eyes on it is still welcome.

Is this ready to go (followed by #6221)?